March 19, 2018
Original Average \(\approx 60\)%
Similar across tutorial sections
\[ScaledPoints = (MidtermPoints - 27.4) \cdot 0.715 + 32.6\] Original Average: \(59.9\)% New Average: \(70.9\)%
It is possible
If \(X \rightarrow Y\):
It is possible that \(X,Y\) are correlated, without causation
| i | \(Y_i^0\) | \(Y_i^1\) | \(Y_i^1 - Y_i^0\) | \(X_i\) | \(W_i\) |
|---|---|---|---|---|---|
| 1 | 5 | 9 | 4 | 1 | 1 |
| 2 | 4 | 8 | 4 | 1 | 1 |
| 3 | 3 | 7 | 4 | 0 | 0 |
| 4 | 2 | 6 | 4 | 0 | 0 |
True causal effect of \(X\): \(4\).
Apparent causal effect of \(X\):
\[\frac{9+8}{2} - \frac{3+2}{2} = \frac{12}{2} \neq 4\]
| i | \(Y_i^0\) | \(Y_i^1\) | \(Y_i^1 - Y_i^0\) | \(X_i\) | \(W_i\) |
|---|---|---|---|---|---|
| 1 | 5 | 9 | 4 | 1 | 1 |
| 2 | 4 | 8 | 4 | 1 | 1 |
| 3 | 3 | 7 | 4 | 0 | 0 |
| 4 | 2 | 6 | 4 | 0 | 0 |
but \(W\) makes effect of \(X\) look too big
\(W\) is a confounding variable
We observe \(X,Y\) correlated…
Comparative Method:
If causal claim that \(X \rightarrow Y\), then:
We generate empirical prediction:
If we observe two cases to be the same in all relevant respects except for value of \(X\), then we should observe that the two cases differ in the value of \(Y\)
Comparative method
Correlation
Two approaches to make correlation work like the comparative method
An experiment is design-based solution:
This extrapolates comparative method to correlation.
The goal is to:
conditioning: the process of holding other variables constant while looking at relationship between \(X\) and \(Y\).
How to think about adjustment/conditioning:
It is like:
It is not this process EXACTLY, but similar
Does UN peace-keeping prevent re-occurrence of civil war?
Exposure to UN peace-keeping at the end of a conflict causes countries to experience longer periods of peace
Correlation is insufficient. Many variables may cause UN intervention AND ability to have durable peace
Look at correlation between Peacekeeping and Peace within groups of cases that have same values of:
DOES NOT eliminate the causal link between confounding variables and peacekeeping by design. We adjust our correlation to account for confounders.
Example (1)
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | \(\mathbf{20}\) | 10 |
| treated | 20 | 10 | 1 | 0 | \(\mathbf{20}\) | 10 |
| treated | 15 | 5 | 1 | 1 | \(\mathbf{15}\) | 10 |
| untreated | 20 | 10 | 0 | 0 | \(\mathbf{10}\) | 10 |
| untreated | 15 | 5 | 0 | 1 | \(\mathbf{5}\) | 10 |
| untreated | 15 | 5 | 0 | 1 | \(\mathbf{5}\) | 10 |
What is the true effect of peace-keeping?
\[\frac{10 + 10 + 10 + 10 + 10 + 10}{6} = 10\]
\(10\) years more peace.
What is the effect we see with unadjusted or unconditional correlation?
(Average in peacekeeping - Average in no-peacekeeping)
Example (1)
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | \(\mathbf{20}\) | 10 |
| treated | 20 | 10 | 1 | 0 | \(\mathbf{20}\) | 10 |
| treated | 15 | 5 | 1 | 1 | \(\mathbf{15}\) | 10 |
| untreated | 20 | 10 | 0 | 0 | \(\mathbf{10}\) | 10 |
| untreated | 15 | 5 | 0 | 1 | \(\mathbf{5}\) | 10 |
| untreated | 15 | 5 | 0 | 1 | \(\mathbf{5}\) | 10 |
\[\frac{20 + 20 + 15}{3} - \frac{10 + 5 + 5}{3} = \frac{35}{3} \neq 10\]
We have upward bias: it looks like peace-keeping causes more peace than it does
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | \(\mathbf{1}\) | \(\mathbf{0}\) | 20 | 10 |
| treated | 20 | 10 | \(\mathbf{1}\) | \(\mathbf{0}\) | 20 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 20 | 10 | \(\mathbf{0}\) | \(\mathbf{0}\) | 10 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | 1 | \(\mathbf{0}\) | 20 | 10 |
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | 1 | \(\mathbf{0}\) | 20 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | \(\mathbf{20}\) | \(\mathbf{10}\) | 0 | \(\mathbf{0}\) | 10 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
Result?
Conditioning! Calculate effect of peacekeeping where Intense War is \(1\)
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| untreated | 20 | 10 | 0 | 0 | 10 | 10 |
| treated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{1}\) | 1 | \(\mathbf{15}\) | 10 |
| untreated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
| untreated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
Effect of peacekeeping where war intensity is high (1):
\[\frac{15}{1} - \frac{5 + 5}{2} = \frac{10}{1} = 10\]
… and where war intensity is low (0)?
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| untreated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{0}\) | 0 | \(\mathbf{10}\) | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
Effect of peacekeeping where war intensity is low (0):
\[\frac{20 + 20}{2} - \frac{10}{1} = \frac{10}{1} = 10\]
Within the groups of cases defined by values of war intensity:
Why does this work?
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | 0 | 20 | 10 |
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | 0 | 20 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{0}\) | 0 | 10 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{1}\) | 1 | 15 | 10 |
| untreated | 20 | 10 | 0 | 0 | 10 | 10 |
| untreated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{0}\) | 1 | 5 | 10 |
| untreated | \(\mathbf{15}\) | \(\mathbf{5}\) | \(\mathbf{0}\) | 1 | 5 | 10 |
(Gilligan & Sergenti 2008)
Look at correlation of peacekeeping and peace after conflict:
(Lower proportional hazards => war less likely)
(Gilligan & Sergenti 2008)
Conditioning on other factors (holding them constant):
Peacekeeping makes war 85% less likely to re-emerge
The presence of guns causes an increase in violent crame
The firearm ownership rate causes an increase in the violent crime rate
Correlation between gun ownership and homicides is nearly \(0\).
Buyer beware. Unless stated otherwise…
Holding county-attributes constant:
Unadjusted correlation between guns and homicide: biased downward
Let's see what happens when this goes wrong…
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 18 | 8 | 0 | 0 | 8 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
What is the true effect?
\[\frac{10 + 10 + 10 + 10 + 10 + 10}{6} = 10\]
What is the unconditional effect of UN peacekeeping?
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| treated | 20 | 10 | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| treated | 15 | 5 | \(\mathbf{1}\) | 1 | \(\mathbf{15}\) | 10 |
| untreated | 18 | 8 | 0 | 0 | 8 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
What is the unconditional effect of UN peacekeeping?
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 18 | 8 | \(\mathbf{0}\) | 0 | \(\mathbf{8}\) | 10 |
| untreated | 15 | 5 | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
| untreated | 15 | 5 | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
What is the unconditional effect of UN peacekeeping?
\[\frac{20+20+15}{3} - \frac{8+5+5}{3} = \frac{37}{3} \neq 10\]
Let's try to condition on War Intensity
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| treated | 20 | 10 | \(\mathbf{1}\) | 0 | \(\mathbf{20}\) | 10 |
| untreated | 18 | 8 | \(\mathbf{0}\) | 0 | \(\mathbf{8}\) | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| treated | 20 | 10 | 1 | 0 | 20 | 10 |
| untreated | 18 | 8 | 0 | 0 | 8 | 10 |
| treated | 15 | 5 | \(\mathbf{1}\) | 1 | \(\mathbf{15}\) | 10 |
| untreated | 15 | 5 | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
| untreated | 15 | 5 | \(\mathbf{0}\) | 1 | \(\mathbf{5}\) | 10 |
When War Intensity is "low" (0):
\[\frac{20 + 20}{2} - \frac{8}{1} = 12 \neq 10\]
When War Intensity is "high" (1):
\[\frac{15}{1} - \frac{5+5}{2} = 10\]
We still have bias in the "low" war intensity group… why?
| \(Peace_i^{UN}\) | \(Peace_i^{noUN}\) | \(UN_i\) | \(WarIntense_i\) | \(Peace_i^{Obs}\) | \(Effect_i\) | |
|---|---|---|---|---|---|---|
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | \(\mathbf{0}\) | 20 | 10 |
| treated | \(\mathbf{20}\) | \(\mathbf{10}\) | \(\mathbf{1}\) | \(\mathbf{0}\) | 20 | 10 |
| untreated | \(\mathbf{18}\) | \(\mathbf{8}\) | \(\mathbf{0}\) | \(\mathbf{0}\) | 8 | 10 |
| treated | 15 | 5 | 1 | 1 | 15 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
| untreated | 15 | 5 | 0 | 1 | 5 | 10 |
… even after conditioning?
Within the "low" war intensity group:
If we fail to condition on all confounding variables, still have bias
If we …
… then adjustment/conditioning might not work
Adjustment/conditioning: adjusts correlation of \(X\) (cause) and \(Y\) (outcome) to account for confounding variables
This works…
design-based solutions to spurious correlation or confounding:
Whether using correlation or comparative method:
What does this do? Comparisons like this:
Do minimum wage laws actually improve wages?
Correlate unemployment with minimum wage laws across countries
Countries with different minimum wages likely differ in many ways:
Compare minimum-wage laws and unemployment across provinces within the same country
Compare minimum-wage laws and unemployment across border-counties within the same country, on the same provincial border
Provinces/states might also differ from each other in many ways:
Does the structure of race in American cause African Americans to have lower income?
Look at incomes of African Americans versus whites
Lots of historical causes of lower African American incomes, hard to find contemporary effects.
Economists at Stanford, Harvard, Census:
within neighborhood comparison:
Similar historical background:
Similar historical context:
Hard to match all of these attributes individually, choice of comparison helps
If we extend logic of similar cases:
What does comparison of case to itself do?
Correlate speed enforcement and fatalities across states
In 1956, Connecticut government responded to traffic deaths by increasing state police enforcement of speed limit and stronger fines/sentences
We can compare Connecticut pre- and post- "crackdown"
Can police leadership change officer behavior?
Can police leadership cause officers to reduce racial profiling
In March 2013, NYPD leadership suddenly mandated that officers had to provide a justification for SQF stops when submitting their reports.
Measure police profiling using "success rate" of weapons stops
Argument for design assumptions replaces argument for including all variables
Looking at same case over time:
Looking at similar cases at the same time:
Can we design a comparison that combines these approaches?
Yes, if we make some assumptions.
difference in difference: difference (between cases) in difference (within cases)
Consider 2 states \(State_A\) and \(State_B\) at two times \(Time_0\) and \(Time_1\)
So:
| \(Time_0\) | \(Time_1\) | First Difference | |
|---|---|---|---|
| \(State_A\) | \(7.3\) | \(7.8\) | \(0.5\) |
| \(State_B\) | \(6.2\) | \(6.8\) | \(0.6\) |
| Second Difference | \(-0.1\) |
Consider 2 states \(State_A\) and \(State_B\) at two times \(Time_0\) and \(Time_1\)
| \(Time_0\) | \(Time_1\) | First Difference | |
|---|---|---|---|
| \(State_A\) | \(5.3\) | \(5.4\) | \(0.1\) |
| \(State_B\) | \(2.1\) | \(2.7\) | \(0.6\) |
| Second Difference | \(-0.5\) |
first difference: within cases over time holds fixed all unchanging confounding variables for each case
second difference: between cases over the same time holds fixed all shared trends over time
In terms of counterfactuals:
How do we know that trend in untreated case is counterfactual for trend in treated case?
Does Minimum Wage increase affect unemployment?
Assumption that "treated" and "untreated" trends are counterfactuals
Does increase in gun ownership lead to more violence?
Do states with no waiting periods…
Assumptions appear to hold:
What do we do if we can't find another case that we think has the same trend as the case with treatment?
In 1995, Connecticut passed law requiring permit-to-purchase a handgun.
"Frankenstein" Connecticut has same pre-law trends in:
as the Real Connecticut
Compared to the "control", Real Connecticut:
Gun control causes fewer overall homicides, by stopping firearm homicides
Compare groups of cases that differ in exposure to cause at random by nature
standard: lotteries, simple random process of assigning treatment
regression discontinuity: fancy phrase for when treatment is assigned at a cut-off
instrumental variables: antecendent variable to our cause is randomly assigned
standard natural experiments:
In 1854, Cholera broke out again…
This time, Snow was ready to get serious.
Noticed a difference between two water suppliers:
Importantly:
Snow's contention is:
regression discontinuity natural experiments:
Looking at MLAs in India between 1961 and 2000
What can we compare?
Why?
Places where Congress MLA barely wins:
instrumental variable natural experiments:
We isolate changes in our cause \(X\) due to randomness in antecedent variable \(Z\)
Does increasing economic growth decrease likelihood of civil war?
Instead of adjusting for possible confounders…
Different solutions, different answers: